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A general theory is presented for evaluating the error performance of a 
digital FM system in the presence of additive noise. The digital system con- 
sidered is a conventional one employing a voltage-controlled oscillator as the 
modulator and a limiter -discriminator followed by a low-pass fdter as the 
demodulator. Because of the nonlinear nature of the demodulation process, 
no adequate analytical techniques have been available to provide a satisfac- 
tory treatment. Adopting the notion of "clicks" used by S. 0. Rice to study 
threshold effects in analog FM systems, we have succeeded in evolving a 
theory capable of predicting performance for a wide range of applications. 
While our theory reinforces some previously derived results for binary and 
for narrow-band systems, the remits obtained here are not confined to these 
situations. In particular, the inefficiency of the FM discriminator as a de- 
tector for a large number of orthogonal signals is quantitatively evaluated, as 
well as the role of the post-detection filter. Some qualitative aspects of the 
error -causing mechanisms discussed in the paper are general, but quantita- 
tive results are confined to additive Gaussian noise and large signal-to- 
noise ratios. 

I. INTRODUCTION 

Theoretical investigations of FM receivers with analog input signals 
date back to J. R. Carson and T. C. Fry, 1 and to M. G. Crosby. 2 These 
investigators and others that followed them 3 ' 4,5 were primarily concerned 
with the signal-to-noise (8/N) transfer attainable in FM receivers and 
the determination of threshold effects. Recently S. O. Rice, 6 and previ- 
ously J. Cohn, 7 attacked the threshold problem in FM receivers from a 
fresh point of view by using the notion of "clicks." It has been observed 
that when the noise at the input of an FM receiver is increased beyond 
some value, the receiver "breaks," that is, for a given (S/N) at the in- 
put, a much poorer (S/N) at the output is measured than would be pre- 
dicted from a linearized analysis of the receiver. Before the breaking 
point, clicks are heard in the output of an audio receiver. As the input 
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noise is further increased, the clicks merge into a sputtering sound. 
Rice's approach is to relate this breaking point with the expected num- 
ber of clicks per second at the output due to the added noise at the in- 
put. 

While in analog application the criterion of (S/N) transfer is satisfac- 
tory, in digital data transmission it does not by itself furnish an adequate 
performance criterion. Usually performance is judged in terms of error 
rates which cannot be predicted from the (S/N) transfer for nonlinear 
receivers. The error rate clearly depends on the statistical distribution of 
the output noise. In good systems, the errors are very infrequent and are 
associated with rare peak noise conditions. The statistical structure of 
the occurrence of infrequent noise peaks and the manner in which they 
cause errors in FM receivers is the main subject of this paper. Some 
previous investigations of these effects have been carried out. For ex- 
ample, Bennett and Salz 8 have analyzed binary FM systems, including 
the effects of distortion. They derived formulas for the error rate without 
including the post-detection filter in their model. Since the error rates 
that they obtained for a well-designed binary system were close to the 
optimum obtainable for any receiver, they were able to conclude that the 
neglect of this filter was justified. Formulas are also available 9,10 for the 
probability distribution function of the instantaneous frequency of sig- 
nal plus noise at the input to the post-detection filter for JV-ary FM, but 
these equations are not very useful in predicting the performance of a 
practical FM system since the task of relating this distribution to the 
distribution at the output of the post-detection filter is apparently 
untractable. In a recent paper, Salz 11 considered a multilevel FM narrow- 
band digital communications system where he included the post-detec- 
tion filter in his analysis. However, the results assume that the post-de- 
tection filter did not perform significant selective processing of the 
detected signal. 

In this paper, we shall develop a general theory from which the per- 
formance of FM receivers with arbitrary processing gain may be pre- 
dicted. We shall view the conventional FM receiver, described in Section 
II, as a device for detecting digital signals and examine its properties in 
detail. In Section III, after approximating the post-detection filter by an 
ideal integrator, we show how clicks enter the problem.* Our assumptions 
and the ensuing mathematical model of the stochastic output are also 
stated there. The following section supplies the considerable amount of 



* Cohn, Ref . (7). has also mentioned the application of the concept of clicks to 
explain errors in digital FM. Further, D. Schilling of Brooklyn Polytechnic Insti- 
tute has called to the authors' attention that he is also investigating the relation- 
ship between clicks and error rates in FM. 



DIGITAL FM 1513 

mathematical detail needed to quantitatively substantiate the work of 
Sections V through VII. In particular, the notion of clicks will be used to 
explain the poor performance (compared to ideal) of this receiver to de- 
tect a large number of orthogonal signals. This phenomenon has also been 
mentioned by Wozencraft and Jacobs. 12 Another result of the present 
paper is to establish conditions under which the previous analyses reli- 
ably predict the performance of actual FM systems. The work of Refs. 8 
and 1 1 will be supported and it will be shown that for multilevel wide- 
band systems the post-detection filter cannot be ignored. Finally, in Sec- 
tion VIII a discussion is given to suggest circumstances under which suc- 
cessive clicks will not be independent and an instructive example is 
given showing how this renders ineffective the additional selective filter- 
ing possible at the input when the frequencies are very widely spaced. 

II. THE DIGITAL FM SYSTEM 

A digital FM signal is readily produced by changing the frequency of an 
oscillator in response to a digital baseband signal. The voltage or current 
at the output of such an oscillator may be represented as 



S(t) = A cos a>4 + f s(t')dt' + 



(1) 



where A is a real amplitude, w c the angular center frequency of the oscil- 
lator, and is an initial phase angle. The digital information-bearing 
signal s(t) is taken to be a piece-wise constant function of time represent- 
able as a random time series of the form 

n=°o 

s (t) = u> d £ a n g(t - nT), (2) 

n=0 

where [a n , n = 0, 1, ■ ■ ■ } is a sequence of independent and identically 
distributed integer valued stochastic variables representing the data. 
For example, one might have a n = ±1 with equal probability for binary 
systems. The function g{t) is a rectangular pulse of unit amplitude and T 
seconds duration and o> d is a proportionality constant relating frequency 
displacement to baseband signal voltage or current. The spectral proper- 
ties of this FM wave have been extensively analyzed in Refs. 13 and 14. 
Transmission and reception of the FM wave is accomplished as follows. 
The wave S(t) is first processed by a transmitting filter, channel noise is 
added, and the result is processed again by a receiving filter assumed to 
be the inverse of the transmitting one. The signal is then detected via 
the limiter-discriminator and filtered at baseband before being synchro- 
nously sampled at t = nT (using independent timing information) to de- 
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Fig. 1 — Block diagram of a digital FM receiver. 

termine sequentially the values of { a„ } . We have illustrated these opera- 
tions in block diagram form in Fig. 1 . A detailed description of the blocks 
shown is given in Ref. 15. We shall state here in mathematical terms the 
assumed operation of the limiter-discriminator. Let the input to the 
limiter be written in terms of in-phase and quadrature components as 



where 



and 



x'(t) cos co c t — y'(t) sin u c t = R(t) cos [w c t + <p(t)\, (3) 

*(*) - vV(/)] f + WdW (4) 

(5) 



<p(t) = tan- 1 y'(t)/x'(t). 
Then the output of the discriminator is taken to be 
dp x'{t)y'(t) - y'{t)x\t) 



dt 



W(f)] 2 + [y'iOY 



(6) 



where the dots denote differentiation with respect to time. The post- 
detection filter acts upon the quantity (6). 

III. FORMULATION OF THE PROBLEM AND A MATHEMATICAL MODEL 

We approximate the low-pass filter as an ideal integrator whose im- 
pulse response is unity for T' seconds and zero afterward. The duration 
T' is taken equal to the signaling time T and so no intersymbol interfer- 
ence occurs at the sampling times for a wave described by (1) and (2). 
The results obtained with this particular choice of filter should be repre- 
sentative of the results one would obtain with any low-pass filter of simi- 
lar bandwidth. 

The sampled output q of the discriminator low-pass filter output is 
given by (7) 



nT I 

, J X 



(t)y'(t) - y'(t)x'(t) 
x'\t) + y*(0 



dt. 



(7) 
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The in-phase and quadrature components occurring in (7) are now not 
those of the pure FM wave (1), but have the analogous components of 
zero mean noise added in as well. One may, by use of a rotating coordinate 
system, equally consider 



q = q — a„wdT 



-f 



x(t)y(t) - y(t)x(t) 



x\t) + y\t) 



-dt 



(8) 



where y(t) is now a zero mean quadrature noise process, while x(t) is an 
in-phase noise process with mean A, the amplitude of the noise-free re- 
ceived FM wave. We now proceed formally with (8), denning a quantity 



r(t) = y(t)/x(t). 



(9) 



rr(T) 

•MO) 1 



(10) 



Equal ion (8) is then rewritten as a path integral 

In (10) we have written d<p = d(tan -1 y/x), but of course we do not mean 
that <p is evaluated using some fixed branch of tan -1 y/x since this would 
give <p as a single valued function of y and x and would not allow for the 
fact that as we circle once about the origin in the ay-plane <p increases by 
2tt. The noise processes y(t) and x{t) wander about the .r?/-plane (see 
Fig. 2), usually staying close to their mean values but occasionally tak- 
ing large excursions and encircling the origin. Each infinitesimal portion 
of the path contributes an amount d<p volts to the output and all these 
small amounts from all the small portions of the path must be added to- 
gether to form the total contribution q. It is easy to see that q depends on 
the path taken, not just on its endpoints. A simple mathematical reason 
for this is that the transformation (9) is undefined whenever x(t) = 0. 
Further, the paths taken in the .ry-plane are random, and q is therefore, 
a random variable with some probability density related to the statistics 



[i(o),y(o)] 




[x(T),y(T)] 



Fig. 2 — A possible path in the xy-p\ane traced by the noise from t = to t = T. 
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of r(t). Unfortunately, this probability density is not determined solely 
by the elementary statistics of r(t). As will be seen, in addition to the 
elementary statistics of r(t) the distribution of its singularities on the time 
axis enters the picture. The singularities of r(t) are determined by the 
zero-crossings of x(t). Thus, the behavior of FM receivers is intimately 
related to the structure of the zero crossings of the added noise. 16 

To see how to handle the situation, visualize the following hypotheti- 
cal state of affairs. Suppose for ^ t ^ T we have that y(t) > 0, and 
that x(t) is positive for a while, decreases once through zero at t = to , 
and then remains negative. A possible plot of r(t) versus t over the time 
interval is then shown in Fig. 3. For this particular path one has 

f dr i r m dr r dr r (f) dr nn 

q ■/,«.) 1 + r 2 + ■/-» 1 + r 2 J_» 1 4- r 2 + J rW 1 + r 2 ' ^ ; 

In (11) the straightforward interpretation of the integrals is meant. 
Evaluating the infinite integral one obtains for this path 

q = t + tan -1 r(T) - tan" 1 r(0), 

where tan -1 x means the principal value inverse tangent function, 
| tan -1 x | ^ 7r/2. In general, one has the result that 

q = tan- 1 r(T) - tan" 1 r(0) + n(T)v, (12) 

where tan -1 x again has the principal value interpretation and n{T) is 
an integer (which may be positive, negative, or zero) which is related to 
the number of times x(t) vanishes in the interal T and to the sign of y(t) 
when x(t) vanishes. For large signal-to-noise ratios it is clear that if 
x(t) vanishes by going to zero from the positive side that it will almost 
immediately be followed by another vanishing of x(t) in the other direc- 
tion. If ?/(/) has not changed, the contribution of the "return trip" to 
n(7') will cancel the contribution from the previous crossing of the y-axis. 
On the other hand, if y(l) does change sign so as to cause an encircling 
of the origin then the contribution to n(t) will be the same as the previous 
crossing. The net contribution to n(T) of a number of paths is shown in 
Fig. 4. The paths which have An = ±2 are immediately recognized as 
the "clicks" discussed by Rice. 6 The "clicks" are not the only contribu- 
tion to n(T) however. There is also a contribution because of the fact 
that at t =s 0, when our process begins, we may be in the middle of a 
large noise fluctuation and be over in the left-half plane. Immediately 
afterwards, at t = 0+, we will experience a contribution of ±1 to n(T) ; 
a similar situation may prevail at time t = T, when a possibility exists of 
stopping the process immediately after we have crossed over to the left- 
half plane. We will show later that for large signal-to-noise ratios, these 
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Fig. 3 — A possible sample function of r(t). 

end-effects may be neglected because they occur with a probability that 
is asymptotically negligible compared with the probability of a click. 

An important fact to observe before proceeding with the analysis is 
that q can be decomposed into the sum of three random variables. The 
first two random variables appearing in (12) are continuous and bounded. 
Their probability densities are related to the elementary statistics of 
x(t) and y(t). The third random variable is a discrete one, whose proba- 
bilities are determined from the probabilities of zero-crossings of x(t) 
and y(t). 

The remarks made thus far about the effect of noise on FM reception 
have been general ; no assumptions have been made about the statistical 
nature of the additive disturbance. In order to obtain quantitative re- 
sults some definite assumptions are necessary. For the remainder of the 
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Fig. 4 — Net changes An in n(T) caused by some typical paths in the xy -plane. 
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paper we shall set ourselves the task of studying the structure of the 
probability distribution of q when the input noise statistics are those of a 
Gaussian process having a symmetric spectral density about the carrier. 
From these distributions we determine the error rates as a function of the 
pertinent system parameters. 

No attempt will be made in this paper to derive an exact probability 
density for the random variable q. This is not a mathematically tractable 
problem since it requires knowledge of the probability distribution of 
zero-crossings of random processes. This by itself has been an area of 
investigation for many years without too much success. The probability 
distribution of the zero-crossings of most elementary random processes 
is not currently known. 

In order to permit an analysis of the model two assumptions are made, 
both of which we feel are quite reasonable. These two assumptions taken 
together state that the three random variables that determine q via 
(12) are all independent. We separate this statement into two assump- 
tions because their individual justification stems from two different 
physical arguments, one having to do with bandwidth and the other 
with signal-to-noise ratio. The first assumption states that tan -1 r{T) 
and tan -1 r(0) are independent. For a flat Gaussian noise input this will 
be a good approximation if T ^ 1/W, where W is the input noise band- 
width. Since T is also the signaling interval, and the correlation func- 
tion of the input noise has its first zero at t "^ 1/W, the motivation for 
this assumption is clear. The second assumption, somewhat harder to 
justify, states that n(T) is independent of the previous two random varia- 
bles, and the clicks, which comprise n(T), are independent from one 
another. This is clearly an assumption expressing an intuitive feeling 
that the clicks occur rarely and of sufficiently short duration. In general, 
they will be rare if the signal-to-noise ratio is large, and short if the 
bandwidth satisfies W ^ \/T as required above. 

These two assumptions plus the identification of crossings of the nega- 
tive x-axis by the moving point in the .cy-plane (as calculated by Rice) 
with the occurrence of a click shall constitute our working model of the 
output noise. An indication of how this model must be modified if the 
input noise spectrum is not relatively flat is given in Section VIII. 

IV. THE BASIC DISTRIBUTIONS 

Let y be a Guassian variable of zero mean, variance a 2 , and x be another 
independent Gaussian variable of mean A, variance a 2 .* Then the den- 

* Recall that even though our x(t) and y(t) are not independent processes be- 
cause the noise spectrum will not be symmetrical about (w c + On&>d), they are inde- 
pendent variables. 
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sity p(<p) where tan ip = y/x and <p has the full range of 2w is well known 
and is given by Bennett, 17 

_, , exp ( — p) . , . /p _ / • 2 ., 

p($) = — *-£ — — + $ 4/ ~ cos * ex P ( _ P sm W 



2t " T r ■ ■ (13) 

•[1 + erf (Vp cos£)], 

where p - 4 2 /2<r 2 . 

One fact which is implicitly contained in (13) is the probability P L 
of finding the signal point in the left half of the zy-plane. However, an 
easier way to obtain P L is as follows: 

A exp (— p) 

P. = Pr(s < 0) - J erfo ^ ~ ^^ . ( 14) 

Equation (14) will be of use in the arguments used to discard the "end 
effects" at t = and t = T spoken of earlier. Equation (13) also im- 
mediately yields the probability density p(<p) for <p = tan -1 (y/x), 
— r/2 < tp < 7r/2. Indeed, we have 

p(<p) = p(ip) + p(<p + 7r) 

exp (— p) , /p / . 2 \ « / /- \ *• ' 

= — : — + i / - cos <p exp ( — p sin <p) erf ( Vp cos <p) 

it V IT 

for | tp | ^ tt/2. 

Suppose ^i and p 2 are two independent angles which have the density 
(15), and define an angle $ = ip\ — <p% , | $ | ^ t. It will be of interest for 
us to determine the probability P v that <l> exceeds some angle y> > 0, 
i.e., we would like to determine 

/(r/lt-f rx/2 

<fa \ rhipMvM, f > o. (16) 

In general, one is unable to perform these integrations exactly, but since 
discussion has already been limited to the large S/X region, little will be 
lost if we make use of this in simplifying the evaluation of (16). The 
asymptotic evaluation is carried out in detail in the Appendix; we 
distinguish three cases:* 



Case I; < <p < v/2: 

1 cot dp 2) exp [-2p si n 2 .: 2 
\Z8tt vcos ^ s/ , 



P^^^i c -^^^i. (17a) 



* In (17) the symbol "~" is used to denote asymptotic equality; this has also 
been used in (14). Also (17a) and (17c) do not hold if <p gets too close to the end 
points of the appropriate interval. As a rough rule, <p should not be closer than 
1/Vp radians to the end points. 
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Case II;<p = t/2: 



Case III;*) > tt/2: 



(|) exp (-p). (17b) 



exp [-p(l + cos 2 ?)] 

"«. ~ a — 7= 7^~- ~ . ( 17c) 

p 27tv 7r pvp sin y> cos ^ 



The most important characteristic of the result (17) is the dependence 
of the exponent on angle, since for large p the nonexponential factors 
are relatively slowly varying. 

We should remark that for very small angles (15) is well approximated 
by the Gaussian curve 

P ~„„ ( 2 



g(r) = 4/£«p(-V) (18) 

of zero mean and variance l/2p. The difference angle $ would, for very 
small <£, be well approximated by the difference of two independent 
Gaussian variables, each having the density (18). The quantity P v 
calculated on this basis agrees (asymptotically) with the small angle 
approximation of (17a). 

The final item that we discuss in this section is the density of n(T), 
or rather we discuss the density of that part of n(T) that arises from the 
clicks (An = ±2), ignoring An = ±1 contributions. For this we need 
only take over some ideas and formulas from Rice. 6 We have that 
(ignoring An = ±1) 

vn{T) = 2tN(T), (19) 

where N(T) is the number of clicks that occur in time T. Following Rice, 
we assume that all clicks are independent and that those tending to in- 
crease (decrease) <p by 2ir form a Poisson process with rate of occurrence 
N+(N-). In general, with a modulated signal, N+ and N- are not equal. 
The probability density p(z) of z = N(T) is then given by 

/XT \*/2 

v {z) = exp [- (N + + N.)T] J^ 8(s - ft) \^J 

-h(2TVN\N~-); 

as may be shown by forming the discrete convolution of the densities 
of the positive and negative clicks. In (20) &(•) is the Dirac delta func- 
tion and Ik(ix) is the modified Bessel function of integer order k, be- 
having for small p as 18 






(22) 
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'•MzdTih'' (21) 

also 

/_*(*) = h{z). 

The type of modulation that we are concerned with is when the in- 
stantaneous frequency deviates by cod from the carrier* for a time T, 
T being the signaling and processing interval. For this situation Rice 
gives for the average rates JV+ and N- when the noise at the receiver 
input is Gaussian 

N + = *{ Vf* + U [1 - erf Vp + ptf/i*] 

- U exp (~P)[1 " erf (f d \/~p/r)]} 
and 

N- = N+ + U exP (-p). ( 23 ) 

where f 

r = (l/2r)(*/«r) 
o- 2 = var .r = var ?/ 
<r 2 = var £ = var y. (24) 

Under the assumption that fa is positive we have asymptotically 

N +~^7*L + iw) exp [_p(1 + //A ' 2)] 

tf_~2\T + +/«iexp(-p). (25) 

Thus, we see that for large p an ever greater majority of clicks occur in 
the negative direction (f d > 0) and for our purposes of computing error 
rate the clicks in the positive direction may be neglected; i.e., we shall use 



iV + ~0 



for U > 0. (26) 



A/_ ~/j exp (— p) 



* We trust that no confusion will arise between r introduced in (24) and r(t) 
introduced in (9). 

t The case u<i = corresponds to no modulation. Also, for ease of writing, we no 
longer explicitly consider the factor a n ■ 
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For f d < the situation is reversed of course. We note that the effect of 
the clicks on a modulated carrier is to tend to make the measured fre- 
quencies appear closer to the carrier frequency than the transmitted 
frequencies. That is, confining oneself for the moment to only errors 
caused by clicks, frequencies transmitted higher (lower) than the carrier 
will be measured to be at that frequency or a lower (higher) one, when 
the noise is small. 

Since we shall use approximation (26), the distribution (20) for 
z = N(T) may be replaced by the simpler Poisson one, where the proba- 
bility of getting exactly N (negative) clicks in time T is given by* 

r vml exp(-iV-.T)(iV-T)" w . . 

p[N(T)] = p^^j . (27) 

Also the probability of getting M or more clicks is, for large signal-to- 
noise ratios, approximately the probability of getting exactly M clicks. 

V. DISTRIBUTION OF OUTPUT AND PROBABILITY OF ERROR 

Equations (14), (17), (26), and (27) provide the information required 
to calculate the distribution of q, (12). In principle we simply convolve 
the continuous density of [tan -1 r(T) — tan -1 r(0)] with the discrete 
density of n(T)r. In Fig. 5, we have given a qualitative sketch of the 
result, neglecting end effects. This picture is intended to show that the 
density consists of a central lobe about the transmitted frequency ex- 
tending to ±x on each side, which is the density of [tan -1 r(T) — 
tan -1 r(0)], plus identically shaped lobes displaced by integral multiples 
of 2w toward lower frequencies (assuming fa > 0) . These displaced lobes 
are weighted by the probability of getting the appropriate number of 
clicks to effect the displacement. Thus, the lobe occupying the space 
— 2wt ± T is weighted by the probability of getting exactly n clicks in 
time T. For n = the weighting is essentially one, for large S/N. There 
are, strictly speaking, similar lobes and weightings on the opposite side 
as well, but these weights are, for large S/N, negligible compared to the 
corresponding lobe we have drawn. That is to say, the first lobe on the 
right (not shown in Fig. 5) has small probability compared to the first 
lobe on the left, but has a large probability compared to the second lobe 
on the left. Nevertheless, we have neglected to include it because we will 
generally be concerned with probabilities like Pr[ | q — f d T \ > <p], and 
thus corresponding weights are important. We dwell on this point be- 

* We confine ourselves to fd > 0. Exactly analogous consideration apply to fd < 
0. The case/d = occurs if an odd number of frequencies are allowed. 
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Fig. 5 — Qualitative sketch of density of q' (neglecting end effects) for/* > 0. 
The dashed lines are for reference in the text. 

cause it is conceivable that for some practical or conceptual application 
the neglect would not be justified. 

The discussion given above is still not quite correct; it is modified 
when we include end effects. The principle correction that inclusion of 
end effects will cause is to add two more side lobes, one over the interval 
[— 2t,0] and the other over the interval [0,2ir]. The weightings of these 
lobes certainly should not exceed the estimate given in (14), and this 
will be enough to exclude them for our purposes. 

We now apply our results to some typical calculations. Consider the 
case of narrow band* FM (defined by Af d T < tt), where one has J 
equally spaced frequencies of separation A/«j crowded into a bandwidth 
W. The probability of error for any one of the frequencies f (not situated 
at the ends) is the area outside of the interval bounded by lines L 2 and 
L 3 in Fig. 5. If L 2 and L 3 are defined by | q \ — <p then the probability 
of error for such a frequency is, from (17a), 

cot ip/2 exp [-2p sin 2 (<p/2)] 



Pe = —7= 



\/2ir \/C0S<p 



VI 



(28) 



where, if one assumes that the bandwidth W = JAfd , one would take 



tp = 



irWT 
J 



(29) 



Our requirement that AfdT < it implies J > 2 for the narrow-band 
formula to be applicable (assuming WT = 1). Note sin 2 (p/2) is less 
than \, and thus the exponent in (28) is exp [— kp], where k < 1. Now 
the contribution of the clicks to P e is essentially the area Ax, of the 
first side lobe which is by (26) and (27) 



A L =f d Texp(- P ). 



(30) 



* Note the special sense in which the term is nesd here. 

t The P e for a frequency at the end is one-half the expression (28) . 
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But expression (30) is, asymptotically, exponentially small compared to 
(28). Likewise, the area due to the side lobes caused by end effects is 
exponentially small, and the probability of error for narrow-band FM 
is given by (28). The result that the clicks do not asymptotically con- 
tribute to errors in narrow-band multilevel FM lends justification to a 
previous evaluation of this type system by Salz, 11 who considered the 
special narrow-band system with WT = 1. It is both interesting and 
gratifying that this result is in agreement with the result given in Ref. 
11. In a later paper, Salz and Koll 19 report on experimental results which 
agree with the earlier theoretical results. 

Next, consider the asymptotic evaluation of P„ for the case of orthog- 
onal signals; this case corresponds to (Acod)T = ir, and we assume that 
the thresholds are spaced midway between the frequencies. Thus, (for a 
frequency not on the edges) we have that the error probability is given 
by the area outside of that bounded between the lines L\ and L 4 . The 
contribution from the major lobe is, from (17b), 

\ exp (-p). 

In addition, the area of the first side lobe is asymptotically comparable 
to this and is 

f d Texp(-p), 

being weakly dependent on the frequency sent. In fact, for the nth signal 
(J = 2ri) we have for orthogonal signals that 



n J 

2> n = 1 > 2 >'-', 2 - 



The average error rate is then, for orthogonal signals (J of them, J 
even, and equally spaced signals and thresholds), 

P, = (*) exp [-p] + (£)(//2 + 1) exp (-p). (31) 

Equation (31) is indeed a surprising result. The first term of (31) is the 
probability of confusing the transmitted frequency with one of its 
nearest neighbors. The second term is the (average) probability of con- 
fusing it with its second nearest neighbor closest to the carrier. This is 
because the area from (— tt) to (— 37r/2) is, by application of (17b), 
negligible compared to the area from (— 3t/2) to (— 57r/2). Thus, it 
states that for the multilevel scheme considered here (a not unreason- 
able one) one is less likely to confuse a transmitted frequency with its 
nearest neighbors than one is to confuse it with a particular one of its 
second nearest neighbors. We see from (31) that the error rate from the 
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continuous part of the output is comparable to the error rate caused by 
clicks. 

As a final remark about the orthogonal system we see comparing 
(31) and (14) why end effects are neglected again. 

For a final example, consider the wide-band situation where the signals 
are loosely packed in the band; i.e., (Awd)T > it. Now no errors will be 
caused by the continuous part of the output ; only clicks will cause errors. 
If the frequencies are widely spaced a single click may not cause an error; 
several clicks during the time interval T may be required. Thus, suppose 
that the frequencies are spaced so that the phase differences of nearest 
neighbors is (Aud)T = 2mr, n being any positive integer. The probability 
of error will then be the probability of getting n (or more) clicks in time 
T, which from (26) and (27) behaves as 

(/ d 7T exp(-np) > (n/2) n exp(-wp) 

n\ n\ (32) 

^ \ exp ( — np). 

The coefficient in (32) is at least as bad as for the orthogonal case, but 
the important item is the exponent. Superficially at least it appears that 
we have gained in performance by spacing the frequencies widely, since 
the exponential has changed from e~~ p from the minimum orthogonal 
case (AojdT = tt) to e" p . One must realize, however, that one is talking 
about different p's here. The bandwidth for the case under consideration 
is essentially 2n times the minimum orthogonal one and therefore, for 
the same signal power, the nominal value of p has decreased 2n, and one 
has in fact not gained a factor of n in the exponent. In addition to the 
bandwidth penalty, error performance has actually suffered too. 

VI. COMPARISON WITH OPTIMUM 

One can demonstrate how the FM discriminator compares with the 
optimum detector when used to detect orthogonal signals; i.e., when 
Au>dT = ir. It is known that when optimum detection is used for any 
orthogonal set of signals, the (exponential part of the) error rate behaves 
as exp [—E/No], where E is the signal energy (assumed common to all 
J levels) and No/2 is the (two-sided) spectral density of the noise. If we 
let S denote the average signal power, write E = ST, and estimate the 
total bandwidth W for large J by TF = J/(2T), we see that the ideal 
exponent becomes exp [— Jp/2], However, we had seen that, regardless 
of the number of levels, the discriminator error rate for AwdT = it be- 
haves as exp [ — p]. Thus, we have lost a factor of J in the error exponent 
by substituting discriminator detection for matched filter detection. 
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An important conclusion may immediately be drawn concerning the 
performance of conventional FM receivers or detectors of orthogonal 
signals. Our results show that the receiver is indeed inferior in perform- 
ance when compared with the optimum. This fact has been stated by 
Wozencraft and Jacobs 12 and the reasons are clear from our analysis. The 
FM receiver admits too much noise at its front-end which cannot be 
cleaned by the post-detection filter because of the nonlinear anomalies, 
namely the clicks. As a matter of fact, the amount of noise grows in 
direct proportion to the number of orthogonal signals, hence the inferior 
exponent.* The optimum detector is a bank of matched filters. The noise 
power at the output of each filter does not grow with the number of 
signals; it is a fixed constant determined by the bandwidth of the filter, 
which roughly needs be no broader than the symbol rate, 1/T. 

This poor performance of conventional FM receivers when used to 
detect data might be remedied by employing an FM with feedback 
system such as described in Refs. 20 and 21. The physical argument to 
support this contention is often stated as follows. In the absence of the 
feedback loop, the IF filter must be wide enough to pass the total swing 
of the incoming signal. However, since the feedback loop tracks the in- 
coming frequency, this IF filter, whose width determines the noise vari- 
ance, could be narrowed and less noise would be admitted. 

This possibility of making use of FM with feedback to improve the 
error rate in digital systems has been suggested by Wozencraft and 
Jacobs. 12 Unfortunately a mathematical treatment of this difficult 
problem does not exist at present. 

VII. EFFECT OF POST-DETECTION FILTER 

In the previous sections we have discussed in detail the performance 
of an FM discriminator followed by a low-pass filter. The low-pass filter 
was approximated by an ideal integrator whose integration time was 
taken to be equal to the duration of an individual signaling interval. 
Formulas sufficient to evaluate the performance of multilevel FM with- 
out the post-detection filter have recently been developed by Mazo and 
Salz; 10 comparison of the results of the present paper with Ref. 10 will 
show the influence of filtering. 

* Actually, these qualitative conclusions may be arrived at by the Gaussian 
approximation to the output noise. The reason why this works is apparent from 
(31) which gives P c for orthogonal signals. The first term of (31) is not due to clicks 
but arises from the continuous part of the output noise. This is the part that the 
Gaussian approximation would tend to duplicate. The second term of (31) is due 
to clicks and has the same behavior with regard to p. Even if one could keep p 
constant as the number of levels J increased, the factor of J in the click contribu- 
tion to (31) would still degrade performance. 
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Suppose that the angular frequency \j/ is sent and we ask for the proba- 
bility that the observed output is less than z, where (\p — z) > 0. It is 
shown in Ref. 10 that the probability P is essentially given by* (for 
large p) 

«-[-'&$]■ (33) 

Consider the situation for orthogonal signals, or in fact for any signal 
set where the frequency spacing between the individual frequencies is 
fixed. One expects the ratio a 2 /a 2 to increase as the square of the total 
input bandwidth, hence as J 2 , the square of the number of levels. Thus, 
for a large number of orthogonal levels the post-detection filter does 
very well in improving the error performance, changing the error rate 
fromf (roughly) exp ( — p/J 2 ) to cxp ( — p). One would certainly expect 
something like this to be true since, for a large number of levels, the noise 
bandwidth before the post-detection filter is much greater than the 
signal bandwidth at that point. 

Another qualitative effect of the post-detection filter may be noted. 
From (33) we see that the distribution of output noise without the post- 
detection filter depends on the frequency sent, because of the factor 
(z 2 + a 2 /a 2 ) in the exponent; the "spread" of the probability density 
will be roughly twice as great at the ends of the band than at the center, 
and thus without a post-detection filter one would not choose the fre- 
quencies to be equally spaced. We have seen that there is no such de- 
pendence of the error rate exponent on the transmitted frequency when 
the post-detection filter is present. 

VIII. AN APPARENT PARADOX 

At this point we have basically concluded our discussion of error rates 
in digital FM, based in part upon the theory of "clicks" in FM receivers. 
In particular, we have seen in Section VI that even when frequencies 
were widely spaced so that w d T is many multiples of 2t the error per- 
formance did not improve. The reason was noted to be that although the 
distance between frequencies increased, the noise admitted to the system 
increased by a corresponding factor. The latter is predicated on the 
assumption that the input bandpass filter is essentially a flat filter up 
to some cutoff frequency determined by the signal spectrum. It may be 
possible, however, to shape the front-end filter so that increasing the 
frequency separation does not cause a proportionate increase in the 

* Equation (33) represents only the exponential part of P. Also (33) is true [see 
Ref. 10] only if (J, - z)*/[z 2 + & 2 /<r 2 ] < 1. 
t Set (f - z) 2 « (A/) 2 , (a 2 /* 2 ) « / 2 (A/) 2 . 
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noise power admitted. We know that the power spectrum of the trans- 
mitted signal will have peaks at the transmitted frequencies of width of 
the order (1/T). Suppose we have a notch filter then, with transmittance 
peaks at the possible frequencies of the appropriate width. The input 
noise power will be constant and therefore by choosing a large enough 
separation one can force the probability of error to be arbitrarily small, 
contradicting optimality considerations for reception of signals against 
a white Gaussian noise background. 

Before giving what we feel is the correct answer to the stated paradox, 
we wish to explore some other considerations which, on the surface, 
might resolve the paradox without changing the basic assumptions of the 
model. One might first object that our argument was too heuristic; is 
the noise power really constant as the frequency separation increases? 
To answer this we have performed the following calculations. We have 
chosen transmitting and receiving filters so that the FM signal is strictly 
undistorted and then optimized the filters to minimize the variance of 
the noise admitted. This procedure is discussed in Ref. 11, and the results 
depend on the power spectrum of the noise. We then specialize to a 
binary system and, using (48) of Ref. 13 for the spectral density of a 
binary FSK wave train, calculate the noise admitted. The result shows 
that while the noise admitted does, in fact, increase as the frequency 
separation increases, it does so only logarithmically with the separation. 
Thus, the error probability still will decrease to an arbitrarily small 
value as the separation increases and from this point of view the question 
is still unresolved. 

A second consideration is the following. The probability of error that 
Ave have calculated was based on asymptotic approximations to formulas 
given in Ref. 6. The results depended only on the amplitude of the re- 
ceived FM wave and the average noise power a 1 at the input to the 
limiter-discriminator; if one allows transmitting and receiving filters 
the more relevant parameters are the average signal power on the line, 
■Piine, and a 2 . However, the exact formulas of Rice also involve the 
quantity fr 2 which is the average power in the derivative of the noise at 
the input to the limiter-discriminator (after the receiving filter).* Let 
S(ca) be the signal spectral density and F(io) the transmittance of the 
receiving filter. Further, let us insist that the signal at the input to the 
limiter-discriminator be exactly the FSK wave described,! so the trans- 
mitting filter is the inverse of the receiving filter. We then have for a 
white noise background of Not 

* Rice, Ref. 6, uses the parameter r = (l/2ir) (a/a). 

t We emphasize that continuous phase at frequency transition is demanded, but 
nothing more. 

X It is for such a noise background that the optimum results are known. 
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<? = ^° f I F(«) | 2 dco (34a) 

p = ^? f co 2 | F(co) | 2 da (34b) 



When one realizes that the spectrum of an FSK wave decreases at 
infinity like the fourth power of the frequency, 13 (34b) and (34c) imply 
that a 2 and the line power Pi ine cannot both be finite. Thus, suppose b 2 
is finite. The convergence of the integral in (34b) implies that | F(co) | 2 
must decrease at least like l/w 3+E , e > 0. The integral for Pii ne will, 
for large to, look like 



J o> 



3+( , 

— • co aco 



which diverges. Likewise, the assumption of finite line power implies b 2 
is infinite. An infinite a 2 certainly violates the conditions under which 
the asymptotic results of Rice's formulas hold. In particular, these 
formulae show that an infinite a 2 corresponds to an infinite average 
number of clicks per second (assuming such a language is still possible) 
and the FM discriminator will not work, in the strict sense. On the other 
hand, if we choose the evil of infinite line power then perfect performance 
is not surprising. 

While the above theorem about a,&, Pn ne is true from a mathematical 
point of view, it is almost irrelevant from an engineering point of view 
because it involves discussions of infinitely large frequencies, and does 
not really eliminate the paradox at all. We need merely precede the 
limiter-discriminator with a flat filter with a cutoff so high that the 
signal is almost undistorted. Since real discriminators work, this is not 
an unreasonable thing to assume. Now & is finite, and although we may 
have to go to extremely large S/N ratios, the paradox is as entrenched as 
ever. 

The resolution of the problem lies in a reinterpretation of Rice's 
calculation of the average number of crossings of the negative .r-axis. 
We had assumed each crossing corresponds to an encirclement of the 
origin which is independent of all past and future encirclements. This is 
reasonable when the receiving filter is essentially flat across the whole 
received spectrum and the correlation time out of the receiving filter is 
small (' — '1/TF"). However, if the input noise spectrum is chopped into a 
few slits or notches, correlations in the noise being processed in the de- 
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tector will persist for a longer time and multiple encirclements of the 
origin can occur with essentially the same probability that one would 
normally associate with a single large excursion close to the origin. 

To make our arguments more precise we consider a binary situation 
at almost zero rate, i.e., we have very narrow filters Fi and F 2 about the 
frequencies (co c + u d ) and (oj c — ay), respectively. The bandwidth of 
these individual filters is of order 1/T. The noise out of Fi and F 2 can 
be written as 



n x (t) = nu(t) cos (o) c + u<i)t — rii y (t) sin (w c + ou)i 
n 2 (t) = n 2x (t) cos (w c — o> d )t — n 2y (t) sin (w c — oi d )t, 



(35) 



where ni x (t), etc., are independent baseband noise currents. If we assume 
that the frequency (co c + w d ) is being transmitted with amplitude A, 
then in a "coordinate system" following that frequency we have 



where 



x = A + X 
V - Y, 

X = n ix + n 2x cos 2o) d t + n 2y sin 2w d t 
y = niy + n i v cos 2oi d t — n 2x sin 2a>^. 



(36) 



(37) 



A typical portion of the path that the noise traces out in the xy plane 
can be calculated from (36) and (37) and is shown in Fig. 6. Neglecting 
the time variations of ni x (t), etc., which vary on a time scale comparable 
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Fig. 6 — Small portions of some noise trajectories when receiving filter has two 
transmittance peaks. 
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to T, we see the path is a circle centered at (A — n\ x , — ni„), of radius 
Vn 2l 2 + w 2w 2 , and counter-clockwise angular velocity of (2a>d). If <ti 2 
and er 2 2 denote the average noise powers out of F x and F 2 , respectively, 
then the probability P that the circle is appropriately situated with a 
large enough radius to encircle the origin is given exactly by 

P = 2f d 2 a [ 2 exp(-p), (38) 

CI 2 + <r 2 2 

where p = A 2 /2(<ri 2 + o- 2 2 ). For the case of a symmetrical spectrum 
about the carrier (<ri 2 = o- 2 2 ), (38) is comparable to (26). However, our 
circle is rotating with frequency 2f d and will have a constant radius for 
about T seconds; thus, it will complete 2f d T revolutions in time T. As 
the frequencies are spread and notched filters are used the noise indeed 
does not increase proportionally, but the number of multiple encircle- 
ments of the origin that a click will make does increase as the separation. 
Thus, the filter shaping under discussion will affect the statistical struc- 
ture of the clicks, preventing a violation of optimality. 

Note added in Proof. A discussion of the click contribution to the error 
rate has been given very recently by J. Klapper in the RCA Review, 
June, 1966. 

APPENDIX 

Asymptotic Behavior of P v 

We wish to record here an outline* of the details of the evaluation of 
(16) for large S/N so as to obtain the results given in (17). If we set 



p(x) = - exp ( — p) + A/ - cos x exp ( — p sin x) 

* V * (39) 

•erf (Vp cos x), 
then according to (16) the required probability is written as 

Wr/2-v>) r(»/2) 

P, = / dyl dxp(y)p(x). (40) 

•'-(t/2) J (u+f) 

If we define the distribution function 



Pit) = f v(y)dy (4i; 

•'-(r/2) 



* We do not explain the techniques used here for the asymptotic evaluation of 
integrals. The interested reader may wish to consult the subjects "saddle point 
method," "Laplace's method," "Watson's lemma" in Ref. 22. 
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and perform and integration by parts, (40) becomes 

•t/2-v 



P„ = f P(y)v(y + <p)dy. 



(42) 



Our evaluation will be based upon approximating the functions P(y) 
and p(y) when p is large. In particular, from (39) we see 



- cos y exp ( — p sin y), 



(43) 



(44) 



p(y) " 

provided y is not close to ±tt/2. Integrating (43) yields 
P(y) ~ Merf V~p + erf (Vp sin y)], 

which will be a good approximation for large p except when y is near 
— it/2. These exceptional points will receive special consideration. 

As a first example consider the case when <p < t/2. The integrand for 
(42) is shown symbolically in Fig. 7. Consider the contribution first from 
negative y. This is from (42), (43), and (44) 



1 f° - - ^o 

- / c/?/[erf Vp - erf (Vp sin | y |)] \/ p - cos (y + <p) 

2 J-(t/2) V * 

•exp [-p sin 2 (y + <p)). 
Next, approximate erf \/p by unity to obtain 



(45) 





— p (y+y) 


P(f)=i 

v p(y) 




p(o)4 


Wf)*o 

1 1 



Fig. 7 — Symbolic representation of the factors in the integrand of (42) drawn 
for <p < 7r/2. 
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- / dy erfc (Vp sin | y |) 4/^ cos (?/ + <p) 

2, J-U/2) V T 



(46) 
exp [— p sin 2 (y + <p)] 



and use the asymptotic expansion 



erfc x ~ /- exp (— x 2 ). (47) 

VTra; 

The resultant integrand has a saddle point at y = —<p/2, and a routine 
saddle point evaluation will yield (17a) of the text. It is easy to verify 
that the error made by replacing erf V p by unity in (45) creates an 
asymptotically small error. Likewise the neglect of positive y is asymptot- 
ically small for 

f(*/2>-* r(T/2)- v r -1 /,\ 

] o dyP(y)p(y + v) ^ J q dyp(y + v ) fg I - - *>J {-} 

i.(T/2)- v I- 

•exp (- P ) + / dy \/ p - 

h V ir (48) 

•cos (y + <p) exp [— p sin 2 (y + y>)] 
exp [ — p sin 2 <p] 
2\/irp sin ^ ' 

by Laplace's method. For <p < x/2 we have 

2 sin 2 (*>/2) < sin 2 <p 

which proves our point. The addition of the term (l/ir) exp ( — p) in 
(48) provides a strict upper bound to p(y + v) and thus takes care of 
special considerations at the right end of p(y + <p). At the left end point 
of the range of integration, (— v/2), p(y + <p) is still well approximated. 
The function P(y) is, however, approximately 

w f )«al=rf[|+ f ], »«-§. (49) 

Using (49) it is easy to obtain an estimate of the contribution of the 
left end point behaving as exp (— p) and this is asymptotically small. 
This ends our discussion for <p < ir/2. 

We give a somewhat more condensed outline for <p = ir/2. The contri- 
bution of the middle of the range of integration is again approximated 
by (46) with <p — ir/2. Using (47) and (46) immediately evaluates to 

[exp (-p)]/4. 
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Next, consider the error made at the right end point. Equation (43) 
holds to within a strip of order l/\/p from 0, after which p(y + <p) 
behaves like [exp (— p)]/r. Therefore, the error behaves like 

1 exp (-p) 1 

2 tc vV 



which is asymptotically small. 

The left end point error is bounded by 

_ /•' e5eJr*)('+ y) ^sinyexpf-pcos'j,)*,- 5 ^^, 

J_ (t/2 ) 7T \2 / V T T'VlTp 

which is again asymptotically small. 

Our final case is <p > ir/2, and this time end point contributions will 
not be small. The reason is that if one examines the integral representing 
the contribution from the middle of the range of integration, i.e., 

i r(r/2)-v _ f~ 

- I dy[eric Vp sin | y \ - erfc \/p] A/ ?- cos (y + <p) 

2 J-(t/2) V * (50) 

X exp [-p sin 2 (y + <p)\ 

it is exponentially dominated by contributions near the end points. But 
in (50) our approximation to P(y) vanishes faster than the correct P(y) 
at y = — x/2, and our approximation to p(y + <p) vanishes at y = 
(t/2 — <p) while the true p(y + <p) does not. This implies that the asymp- 
totic evaluation of (50) will be asymptotically smaller than the correct 
contributions from the ends of the interval. The contribution from the 
right end is 

1 f lT,2) ~* dy [erf Vp - erf Vp sin | y |] exp ( ~ p) 

2 L (rm , (5i) 

exp [— p(l + cos 2 y)] 
4tt s/ir p V ' p sin <p cos 2 cp ' 

The lower limit of integration (51) is immaterial, as will be the upper 
limit in (52) . For the contribution from the left end we have 

— / dy J + y\ A/- cos (y + <p) exp [-p sin 2 (y + <p)] 

7T J-(x/2) _^ J f IT 



exp — p(l + cos 2 <p)] 
47r VV P Vp sin ^j cos <p ' 

The sum of (52) and (51) yields (17c) of the text. 



(52) 
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